XR Rel-17

 RAN1#103-e

8.14   Study on XR Evaluations for NR (RAN1)

Please refer to RP-201145 for detailed scope of the WI

NOTE: SA4 has ongoing work in the XR area, RAN1 will not address these SA4 aspects but will wait for SA4’s outcome

R1-2009283         On work plan for SI on XR Evaluations for NR           Qualcomm Incorporated

8.14.1     Applications, Traffic Model and Evaluation Methodology

Focus on applications and evaluation methodology including identification of KPIs of interest for relevant deployment scenarios

R1-2007555         XR applications and scenarios         FUTUREWEI

R1-2007561         Discussion on applications, traffic model, and evaluation methodology for XR and Cloud Gaming     Huawei, HiSilicon

R1-2007698         Discussion on XR applications, traffic model and evaluation methodologies        vivo

R1-2007843         XR use cases, evaluation methodologies and traffic model       CATT

R1-2007976         Discussion on applications, traffic model and evaluation methodology for XR     ZTE

R1-2008037         Discussion on XR evaluation and Challenges for NR CMCC

R1-2008198         Applications, Evaluation Methodology, and KPIs for XR          Samsung

R1-2008311         XR evaluations for NR: Applications and Evaluation Methodology        AT&T

R1-2008454         XR Applications, Traffic Model and Evaluation Methodology Apple

R1-2008818         Discussion on traffic models and evaluation assumptions for XR            InterDigital, Inc.

R1-2008896         Applications, Traffic Model and Evaluation Methodology for XR evaluations for NR               Nokia, Nokia Shanghai Bell

R1-2008939         Discussion for study in XR evaluation for NR             LG Electronics

R1-2008967         On Applications, Traffic Model, and Evaluation Methodology for XR and CG               MediaTek Inc.

R1-2009006         Scenarios, Traffic Model and EVM for XR   Intel Corporation

R1-2009041         Discussion on XR application and evaluation methodology      Xiaomi

R1-2009087         XR use cases, traffic modelling and performance measure        Ericsson

R1-2009198         Discussion on study on XR evaluations for NR           NTT DOCOMO, INC.

R1-2009280         Evaluation Methodology for XR     Qualcomm Incorporated

[103-e-NR-XR-02] – Eddy (Qualcomm) and Xiaohang (vivo)

Email discussion/approval for applications, traffic model and evaluation methodology

• 1^st check point: 11/5
• 2^nd check point: 11/10
• 3^rd check point: 11/12

R1-2009812        FL Summary of RAN1 103-e agreements and email discussions on Rel-17 SI on XR evaluations for NR    Moderators (Qualcomm, vivo)

Decision: As per email decision posted on Nov.13^th,

Agreement: XR applications

RAN1 confirms that diverse applications of VR1/2, AR1/2, (XR conference FFS), CG are of interest for study. Potential prioritization/down selection of these applications for evaluation is to be discussed after detailed traffic models and relevant evaluation assumptions are stable.

• FFS: other applications, e.g., XR conferencing

Agreement: Traffic model

Traffic model for DL and UL should reflect various aspects, e.g., various bit rates, variable frame/packet (definition of frame/packet to be clarified with traffic model as necessary) size, and periodicity (how to model jitter is FFS).  RAN1 will strive to conclude on detailed traffic models in the next RAN1 meeting (104-e) where SA4 outcome on traffic model is expected to be available.

• Statistical model is preferred.
• It is preferred traffic model for both UL and DL have a certain degree of variability so thatand the total number of traffic models can be reduced. 
• Note: Taking into account the fact that the decision on traffic models may hold many other crucial decisions, discussion on traffic model in the next RAN1 meeting is prioritized from the beginning.  

Agreement:

Adopt the following deployment for XR/CG evaluations

• Indoor hotspot: FR1 and FR2
• Detailed definition of Indoor hotspot refers to TR 38.913.
• Channel model: InH. Detailed definition of InH refers to TR 38.901.
• Dense urban: FR1 and FR2
• Detailed deployment refers to TR 38.913, where single layer with Marco layer is assumed.
• Channel model: UMi. Detailed definition of UMi refers to TR 38.901.

FFS: Whether to prioritize FR1 for evaluation.

Note 1: When selecting the deployment and evaluation assumptions for XR/CG evaluations, it is up to company to evaluate FR1 or FR2 or both for the frequency range.

Note 2: It does not mean that all applications are evaluated for all the deployment scenarios.

Agreement:

Urban Macro can be optionally reported for XR/CG evaluations only for FR1.

·        FFS: whether Uma is optional or not

·        Following parameters can be assumed.

Agreement:

It is to be further discussed how to prioritize the combinations of deployment scenarios and applications after traffic models for each application are stable.

Agreement:

System capacity is defined as the maximum number of users per cell with at least X % of UEs being satisfied.

• X=90 (baseline) or 95 (optional)
• Other values of X can also be evaluated optionally

Note: The exact ‘satisfied’ requirements will be discussed separately

FFS: how to calculate the percentage of satisfied users across multiple drops of simulations

Agreement:

·        Adopt the simulation assumptions in table 1 as below

Table 1: Simulation assumptions for XR evaluation (Part 1) (updated)

Agreement:

Adopt the following UE distribution for XR/CG evaluation for outdoor scenario

• For outdoor scenario:
• FR1: 80% indoor, 20% outdoor
• FR2: 100% outdoor

Other UE distribution can be evaluated optionally.

Agreement:

Adopt the following TDD configuration for XR/CG evaluation

• FR1:
• Option 1: DDDSU
• Option 2: DDDUU
• FR2:
• Option 1: DDDSU

FFS detailed S slot format

Note: Other TDD configuration or FDD can be optionally evaluated.

Agreement:

Adopt the following BS antenna parameters for indoor scenario for XR/CG evaluation

• FR1:
• 32 TxRU, (M, N, P, Mg, Ng; Mp, Np) = (4,4,2,1,1;4,4)
• (dH, dV) = (0.5, 0.5)λ
• FR2:
• Option 2: 2 TxRU, (M, N, P, Mg, Ng; Mp, Np) = (16, 8, 2,1,1;1,1)
• (dH, dV) = (0.5, 0.5)λ

Other BS antenna parameters can be optionally evaluated

Agreement:

For XR/CG evaluation, adopt the following assumptions for downtilt

• Dense Urban
• FFS: 6 or 12 degree
• Other downtilt can be optionally evaluated.
• Indoor hotspot
• 90° (pointing to the ground)

Other downtilt can be optionally evaluated

Agreement:

·        Adopt the simulation assumptions in table 3 as below

Table 3: Simulation assumptions for XR evaluation (Part 3)

Agreement:

The following aspects are to be discussed after traffic model is stable.

• For the system capacity definition, how to determine whether a UE is satisfied or not is to be deferred until the exact traffic model along with how to measure E2E user experience is available. Additional metrics to be collected will be further discussed after traffic model is stable.
• Various options for traffic arrival offset among UEs per cell were proposed by companies, e.g., even offset, random offset, no offset. It will be discussed after traffic model is determined.

Agreement:

System bandwidth for XR/CG evaluations are as follows.

• For FR1,
• Baseline: 100 MHz
• Optional: 20/40 MHz (FFS: 200 MHz)
• FFS FR2

Agreement:

For outdoor scenarios, the baseline BS antenna parameters are as follows.

·        FFS FR1,

o   Option 1: 64 TxRU, (M, N, P, Mg, Ng; Mp, Np) = (8,8,2,1,1;4,8)

o   Option 2: 32 TxRU, (M, N, P, Mg, Ng; Mp, Np) = (8,2,2,1,1,8,2)

o   Option 3: 32TxRUs (M, N, P, Mg, Ng; Mp, Np) = (4,4,2,1,1,4,4)

(dH, dV) = (0.5λ, 0.85λ)

·        FR2:

o   TxRU, (M, N, P, Mg, Ng; Mp, Np) = (4,8,2,2,2;1,1)

(dH, dV) = (0.5λ, 0.5λ)

Other configurations can be optionally evaluated.

Agreement:

UE antenna parameters for XR/CG evaluations are as follows

• FR1:
• Baseline: 2T/4R, (M, N, P, Mg, Ng; Mp, Np) = (1,2,2,1,1;1,2), (dH, dV) = (0.5, N/A)λ
• Optional: 4T/4R, 1T/2R, 2T2R
• FFS FR2: down-selection between the next two options. Please indicate if you have preference.
• Option 1 (Follow Rel-17 evaluation methodology for FeMIMO in R1-2007151)
• (M, N, P)=(1, 4, 2), 3 panels (left, right, top)
• (Mp, Np) is up to company. Need to be reported with simulation result.
• Option 2 (from TR 38.802 – developed in Rel-14)
• 4Tx/4Rx: (M, N, P, Mg, Ng; Mp, Np) = (2,4,2,1,2;1,2), (dH,dV) = (0.5, 0.5)λ, the polarization angles are 0° and 90°

Agreement:

BS Tx power for XR/CG evaluations are as follows

• For Indoor hotspot:
• FR1:
• 24 dBm per 20 MHz
• FR2:
• 23 dBm per 80 MHz. EIRP should not exceed 58 dBm
• For Dense urban:
• FR1:
• 44 dBm per 20 MHz
• FR2:
• 40 dBm per 80 MHz. EIRP should not exceed 73 dBm

For system BW larger than above, Tx power scales up accordingly.

Agreement:

UE max Tx power for XR/CG evaluations are as follows 

• FR1: 23 dBm
• FR2: 23 dBm, maximum EIRP 43 dBm

Agreement: Baseline power evaluation methodology

If UE power consumption is agreed as a KPI for evaluation of XR performance over NR,TR38.840 is the baseline methodology potentially with some modifications if necessary.  RAN1 aim to minimize modeling effort. For example, the following aspects can be considered for further discussion but not limited to.

·        FFS whether/how to model UE power consumption for UE tx power other than 0dBm and 23dBm,

·        FFS whether/how to model UE power consumption for UL slots that are not defined in TR38.840

·        FFS whether/how to model UE power consumption for ‘S’ slot

·        FFS whether/how to model UE power consumption for 400MHz in FR2 including scaling rule for FR2 BWP adaption.

·        FFS whether/how to model UE consumption for the corresponding number of Tx antennas

·        FFS whether/how to model the UE power consumption for UE tx power under FR2

Agreement:

• RAN1 continues to discuss evaluation methodologies for UE power consumption and system capacity.
• RAN1 is to discuss whether/how to study/evaluate mobility and coverage at a later stage, e.g., starting from Q1 2021.

8.14.22     Other

R1-2007699         Performance evaluation of XR traffic            vivo

R1-2007842         Potential area of NR enhancement for the support of XR services           CATT

R1-2008316         Initial evaluation results for XR and Cloud Gaming    Huawei, HiSilicon

R1-2008455         Views on enhancements for XR      Apple

R1-2008819         Discussion on potential enhancements for XR             InterDigital, Inc.

R1-2009289         Initial XR performance evaluations Ericsson

R1-2009281        TR skeleton for Study on XR Evaluations for NR  Qualcomm Incorporated

[103-e-NR-XR-01] – Eddy (Qualcomm)

Email discussion/approval for XR TR skeleton till 11/5

Decision: As per email decision posted on Nov.12^th, the latest draft is endorsed, as v0.0.1 in R1-2009707.

Further revised to fix formatting issues in:

R1-2009811        TR38.838 Skeleton for Study on XR (Extended Reality) evaluations for NR               Rapporteur (Qualcomm)

Finally endorsed in R1-2009818 to fix cover page issue (zip file x9811 includes a TR skeleton not consistent with the correct TR skeleton).

 RAN1#104-e

8.14   Study on XR Evaluations for NR (RAN1)

Please refer to RP-201145 for detailed scope of the WI

8.14.1     Traffic Model

R1-2100207        Discussion on applications and traffic model for XR and Cloud Gaming               Huawei, HiSilicon

Proposal 1: Due to limited time and heavy workload, RAN1 study focuses on the 5 applications in current SID, i.e., VR1/2, AR1/2, and CG. Other applications should be first identified and studied in SA4, and SA4 can send LS to RAN1 for further evaluation if necessary.

Proposal 2: For the traffic model of XR and CG,

·        Statistical model is adopted, and the statistical model can be developed based on SA4 outcomes.

·        RAN1 continues to discuss whether P-Trace based traffic model is applicable in RAN1 evaluations or not.

Proposal 3: The following general traffic model is considered for the XR and CG:

·         #M data streams for DL and #N data streams for UL, where each data stream has separate

o    Packet size distribution

o    Packet arrival interval

o    QoS requirement

·         FFS: the value of #M and #N for each XR and CG application

·         FFS: packet size distribution, packet arrival interval, and QoS requirement for each data stream.

Proposal 4: For XR and CG performance evaluation, the frame size is modelled as truncated Gaussian distribution. FFS: mean and variance.

Proposal 5: For XR and CG performance evaluation, periodic traffic with frame arrival interval 1/FPS seconds is considered as a starting point.

Proposal 6: For XR/CG performance evaluation, frame segmentation is not considered for simplicity, i.e., one video frame is modelled as one packet during simulation.

·        Note: Each packet might be further segmented into one or multiple TBs for transmission in physical layer. The number of TBs and the size of each TB are up to radio resource, scheduling, etc.

Decision: The document is noted.

R1-2101137        Traffic Model for XR and CG       MediaTek Inc.

·        Proposal 1: Adopt the proposed traffic model for cloud gaming traffic.

·        Proposal 2: Two traffic models should be considered depending on the location of the XR/CG server (cloud/Edge).

·        Proposal 3: No MTU packet size restriction when the XR/CG server is located at the Edge.

·        Proposal 4: Jitter modelling is required and shall be taken into account in simulations

·        Proposal 5: traffic model shall take into account different traffic types and possibly differentiated frames within the same application, in both UL and DL directions

Decision: The document is noted.

R1-2100055         XR traffic model FUTUREWEI

R1-2100132         Discussion on the XR traffic models for evaluation    OPPO

R1-2100361         XR traffic model CATT

R1-2100476         Discussion on traffic models of XR vivo

R1-2100528         Discussion on Traffic Model for XR evaluations        ZTE , Sanechips

R1-2100555         Discussion on traffic model for XR study     LG Electronics

R1-2100571         Discussion on XR applications and traffic models      InterDigital, Inc.

R1-2100680         On traffic model for XR    Intel Corporation

R1-2100724         On Traffic Model for XR study       Nokia, Nokia Shanghai Bell

R1-2100775         XR Traffic Model Considerations   AT&T

R1-2100879         Discussion on XR Applications and Evaluation Assumptions   Sony

R1-2101101         Discussion on Traffic model for XR evaluation           Xiaomi

R1-2101240         XR Applications and Traffic Models             Samsung

R1-2101314         Traffic model for XR         Ericsson

R1-2101365         Views on XR traffic models            Apple

R1-2101493         XR Traffic Models            Qualcomm Incorporated

R1-2101635         Discussion on traffic model for XR NTT DOCOMO, INC.

[104-e-NR-XR-01] – Eddy (Qualcomm)

Email discussion/approval for traffic model and capacity KPI

• 1st check point: 1/28
• 2nd check point: 2/2
• 3rd check point: 2/4

R1-2102254        Email discussion/approval for traffic model and capacity KPI for XR               Moderator (Qualcomm)

From GTW session on Jan 28^th,

Agreements: RAN1 adopts a parameterized statistical traffic model for evaluation of XR and CG, and KPI with details as shown below (RAN1 strives to agree on the remaining details during RAN1 #104e, based on SA4 input):

•        There are M1 and M2 streams in DL and UL respectively

o   At least adopt the case where M1=1 & M2=1

o   FFS the values of M1 and M2, including the possibility of being application-dependent

•        DL

o   Bitrate for video streaming

§  VR/AR: [60 Mbps (mandatory), 30 Mbps (optional)]

§  CG: [30 Mbps (mandatory), 45 Mbps (optional)]

·        FFS: other optional values

o   Air interface Packet Delay budget (PDB)

§  Air interface delay is measured from the point when a packet arrives at gNB to the point when it is successfully delivered to UE

§  Air interface PDB for video streaming

·        VR/AR: [10ms (mandatory), 20ms (optional)]

·        CG: [15ms (mandatory), 30ms (optional)]

o   FFS: other optional values

o   FFS: Frame-level/IP packet-level modeling for packet arrival, latency measure, etc.

o   FFS: Packet size, including the possibility of varying packet sizes

o   FFS: Packet Inter arrival time including the possibility of modeling jitter

•        UL

o   FFS: Bitrate

o   FFS: Air interface Packet Delay budget (PDB)

o   FFS: Frame-level/IP packet-level modeling for packet arrival, latency measure, etc.

o   FFS: Packet size

•        Per UE KPI

o   Baseline: A UE is declared a satisfied UE if more than X (%) of packets are successfully transmitted within a given air interface PDB. The exact value of X is FFS.

o   FFS: In addition to the baseline, the following additional method is FFS

§  When determining a XR/CG user is satisfied or not, the following factors are considered. FFS how to use those factors. 

·        Packet loss information

·        Packet delay information

·        Some XR/CG source related information if they can be available within RAN, e.g. the mapping between packet and slices or frames and the packet importance

·        Multiple data streams traffic model

o   FFS if there are multiple streams (if adopted)

•        FFS additional aspects not addressed above.

•        Note 1: Companies are encouraged to provide details such as parameters (e.g., mean, STD, etc.), distributions, etc., by analyzing SA4 input, e.g., V/S/P traces

•        Note 2: All FFS points above are to be further discussed in RAN1 #104e

Agreements

•        Statistical traffic model for a single DL video stream for a single UE

o   The statistical traffic model for a single UE for a single DL video stream in Figure 1 is adopted, where a packet is assumed to represent multiple IP packets corresponding to a single video frame for modelling/evaluation purposes, e.g., traffic arrival, packet size, evaluation of latency and reliability.

•        Frame per second (fps) for DL video stream for a single UE

o   60 fps (baseline)

o   120 fps (optional)

o   Other values, e.g., 30, 90 fps can be also optionally evaluated.

•        Average data rate for DL video stream:

o   VR/AR: 30, 45 Mbps @60fps (baseline)

§  30, 60 Mbps @60fps (optional)

§  Note: this is the aggregated data rate when applicable

o   CG: 8, 30 Mbps @60fps (baseline)

§  8, 45 Mbps @60fps (optional)

o   Other values (in combination with fps) can be also optionally evaluated.

•        Truncated Gaussian distribution is used for the packet size distribution of video stream for AR/VR/CG.

o   Other distribution is not precluded.

•        (Working assumption) Parameters of Truncated Gaussian distribution for Packet size (note: these parameter values are those before the truncation)

o   Mean: Derived from average data rate and fps as follows.

§  (average data rate) / (fps for video stream, i.e., # packets per second in our statistical model) / 8 [bytes]

o   STD

§  TBD

o   Max packet size

§  TBD

o   Min packet size

§  TBD

§  FFS whether or not to use this parameter

•        Per UE KPI

o   Baseline: A UE is declared a satisfied UE if more than X (%) of packets are successfully transmitted within a given air interface PDB.

§  The exact value of X is FFS, e.g., 99, 95

·        FFS different values for I-frame and P-frame if evaluation of them is agreed.

·        Other values can be optionally evaluated

•        DL traffic model: video stream

•        (Working assumption) Parameters of Truncated Gaussian distribution for Packet size (note: these parameter values are those before the truncation)

o   Mean: Derived from average data rate and fps as follows. 

§  (average data rate) / (fps for video stream, i.e., # packets per second in our statistical model) / 8 [bytes]

o   STD 

§  [15% of Mean packet size derived above]

§  Note: The above value is an example for further investigation, and is to be revisited potentially with more inputs from companies in RAN1#104-bis-e

o   Max packet size 

§  [1.5 x Mean packet size derived above]

§  Note: The above value is an example for further investigation, and is to be revisited potentially with more inputs from companies in RAN1#104-bis-e

o   Min packet size 

§  TBD

§  FFS whether or not to use this parameter

§  Note: This is to be revisited potentially with more inputs from companies in RAN1#104-bis-e.

•        Jitter for DL video stream for a single UE

o   (Already agreed) Per the agreed statistical traffic model, arrival time of packet k is k/X1000 [ms] + J [ms], where X is the given fps value and J is a random variable. 

o   (Newly proposed agreement) J is drawn from a truncated Gaussian distribution:

§  Mean: [0]

§  STD: [2 ms]

§  Range: [[-4, 4]ms]

·        Note: The values ensure that packet arrivals are in order (i.e., arrival time of a next packet is always larger than that of the previous packet)

§  Note: The above values for mean, STD and Range are working assumption for initial simulations, and is to be revisited potentially with more inputs from companies in RAN1#104-bis-e

·        Air interface PDB for DL video stream 

o   VR/AR: 

§  10ms 

§  Other values, e.g., 5ms, 20 ms can be optionally evaluated. 

o   CG: 

§  15ms

§  Other values, e.g., 10ms, 30ms can be optionally evaluated. 

o   FFS whether or not to have more than one mandatory value

Working assumption: On UL Traffic model and QoS parameters

•        CG/VR: single stream (pose/control)

•        Traffic model for Pose/control

o   Periodic: 4ms (no jitter)

§  Other values can be optionally evaluated.

o   Fixed: 100 bytes (SA4 input)

o   PDB: 10 ms

•        AR

o   FFS

Agreements: On evaluation of multiple streams/flows:

•        FFS the following in RAN1#104-bis-e

o   Whether/how to model and evaluate I-frame and P-frame for both DL and UL, e.g., separate definition of fps, packet size, QoS requirements (e.g., PER, PDB), etc.

o   Whether/how to separately model and evaluate two streams of video and audio/data for both DL and UL

o   Whether/how to model and evaluate FOV (high-resolution) and non-FOV (lower-resolution omnidirectional) streams, e.g., separate definition of fps, packet size, QoS requirements (e.g., PER, PDB), etc

8.14.2     Evaluation Methodology

Including identification of KPIs of interest for relevant deployment scenarios

R1-2100477        Discussion on evaluation methodologies of XR        vivo

·        Proposal 1: The following combinations of applications and deployment scenarios are prioritized for XR/Cloud Gaming evaluation:

o   Indoor hotspot: VR2, CG, for FR1 and FR2;

o   Dense urban: CG, AR2, for FR1 and FR2;

o   Urban Macro: AR2, for FR1.

·        Proposal 2: For a packet that has exceeded the PDB, support to adopt Option 1 as the starting point.

·        Proposal 3: A UE is regarded as satisfied if the packet error ratio measured for it is equal to or less than the given PER.

·        Proposal 4: The following metrics can be considered for XR capacity evaluation,

o   Percentage of satisfied UEs

o   System capacity

o   CDF of packet error ratio

o   CDF of packet latency

o   CDF of user-perceived throughput

o   Resource utilization

·        Proposal 5: Percentage of UEs being satisfied for each drop can be calculated separately, and then averaged over all the drops.

·        Proposal 6: Adopt random offset for modelling traffic arrival offset among UEs per cell.

·        Proposal 7: The user interaction delay can be used as a key metric for uplink capacity evaluation for uplink interaction and pose information traffic.

·        Proposal 8: The number of satisfied users for interaction and pose information is defined as the maximum number of users per cell for which the A%-tile user interaction delay is equal to or less than the uplink PDB, where the threshould A% should be discussed and determined, when only interaction and pose information are modelled in uplink.

·        Proposal 9: The evaluation methodologies and performance metrics used for downlink are reused for uplink video traffic.

·        Proposal 10: When evaluating the power consumption performance, the capacity performance should be jointly considered to show the potential trade-off between them.

·        Proposal 11: For power consumption evaluation, the simulation assumptions and capacity performance metrics for capacity evaluation can be reused.

·        Proposal 12: For power consumption evaluation, the following aspects should be considered:

o   Improving power consumption models for (1) special slots, (2) multiple UL channels in a slot, such as PUSCH, PUCCH and SRS concurrent in a slot, etc.

o   Introducing enhanced power saving techniques, including starting time adaptation for ON Duration of C-DRX, and Rel-16/Rel-17 power saving schemes such as PDCCH skipping.

·        Proposal 13: For XR/Cloud Gaming coverage evaluation, support to reuse the evaluation methodologies in coverage enhancement SI as the starting point.

·        Proposal 14: For XR mobility evaluation, performance metrics should be identified considering impacts on XR performance due to mobility, such as interruption delay, handover failure rate and cell-edge transmission performance.

·        Proposal 15: For XR/Cloud Gaming capacity evaluation, support to agree the remaining simulation assumptions listed in Table 1.

·        Proposal 16: For XR/Cloud Gaming power consumption evaluation,

o   Power consumption performance is evaluated by using power consumption model in TR 38.840.

o   Power consumption performance and capacity performance are evaluated together by considering different C-DRX configurations.

o   Details of C-DRX configurations are reported by companies.

·        Proposal 17: For XR/Cloud Gaming power consumption evaluation, introduce interpolation algorithm for UL power between 0dBm and 23dBm.

·        Proposal 18: For XR/Cloud Gaming coverage evaluation assumptions,

o   Reuse link budget parameters in TR 38.830 as a starting point.

o   Revise some parameters is needed, such as required SINR, number of RBs occupied.

·        Proposal 19: For XR/Cloud Gaming mobility evaluation assumptions,

o   At least the simulation assumptions for capacity can be reused,

o   Other simulation assumptions can be further studied together with evaluation metrics.

Decision: The document is noted.

R1-2100056         XR evaluation methodology            FUTUREWEI

R1-2100133         Discussion on the XR evaluation methodology           OPPO

R1-2100242         Discussion on evaluation methodology for XR and Cloud Gaming         Huawei, HiSilicon

R1-2100362         Evaluation methodology and performance index for XR           CATT

R1-2100529         On XR Evaluation Methodology     ZTE , Sanechips

R1-2100556         Discussion on evaluation assumption for XR study     LG Electronics

R1-2100572         Discussion on Evaluation Methodology for XR           InterDigital, Inc.

R1-2100586         On Evaluation Methodology for XR and CG MediaTek Inc.

R1-2100681         On evaluation methodology for XR Intel Corporation

R1-2100725         Development of the Evaluation Methodology for XR Study     Nokia, Nokia Shanghai Bell

R1-2100776         XR Evaluation Assumptions            AT&T

R1-2101102         Discussion on evaluation methodology for XR services            Xiaomi

R1-2101241         XR Evaluation Methodology and KPIs          Samsung

R1-2101762         Evaluation methodology for XR      Ericsson (rev of R1-2101315)

R1-2101366         Views on XR evaluation methodology          Apple

R1-2101494         Evaluation Methodology for XR     Qualcomm Incorporated

R1-2101636         Discussion on evaluation methodology for XR            NTT DOCOMO, INC.

[104-e-NR-XR-02] – Xiaohang (vivo)

Email discussion/approval for other evaluation methodology and assumptions

• 1st check point: 1/28
• 2nd check point: 2/2
• 3rd check point: 2/4

R1-2101866        Evaluation Methodology including identification of KPIs of interest for relevant deployment scenarios      Moderator (vivo)

Agreement: Adopt following update for TDD configuration for XR/CG evaluation

• FR1:

• Option 1: DDDSU
• Option 2: DDDUU

• FR2:

• Option 1: DDDSU
• Option 2: DDDUU

Detailed S slot format is 10D:2F:2U. Other S slot format(s) can also be optionally evaluated.

Further clarify that for option 2 for FR1/FR2, there is [2]-symbol gap at the end of third “D” slot of  DDDUU.

FFS whether or not to differentiate the two options (e.g., mandatory vs. optional)

Agreement: For XR evaluation, ideal channel estimation can be optionally evaluated.

Agreements: System bandwidth for XR/CG evaluations are as follows.

• For FR1,

• Baseline: 100 MHz
• Optional: 20/40 MHz, 2*100 MHz with CA

• FR2
• Option 1: 100 MHz
• Option 2: 400 MHz

Companies should report the CA setting if CA is adopted.

Other system bandwidth can also be optionally evaluated.

Agreements:For outdoor scenarios, the BS antenna parameters are as

·        Option 1: 64 TxRU, (M, N, P, Mg, Ng; Mp, Np) = (8,8,2,1,1;4,8)

·        Option 2: 32 TxRU, (M, N, P, Mg, Ng; Mp, Np) = (8,2,2,1,1,8,2)

Company to report the BS antenna parameters for XR/CG evaluation.

Other BS antenna parameters can also be optionally evaluated.

Agreements:For FR2, UE antenna parameters for XR/CG evaluations are as follows.

·        Option 1 (Follow Rel-17 evaluation methodology for FeMIMO in R1-2007151)

o   (M, N, P)=(1, 4, 2), 3 panels (left, right, top)

·        Option 2 (from TR 38.802 – developed in Rel-14)

o   4Tx/4Rx: (M, N, P, Mg, Ng; Mp, Np) = (2,4,2,1,2;1,2), (dH,dV) = (0.5, 0.5)λ, the polarization angles are 0° and 90°

Company to report the UE antenna parameters for XR/CG evaluation.

Other UE antenna parameters can also be optionally evaluated.

Agreements: For XR/CG evaluation, adopt following assumptions for BS height for Urban Macro

Agreements: For Dense urban and Urban Macro, the UE height for indoor UEs is updated as following based on Table 6-1 in TR 36.873.

Agreements: At least for XR/CG capacity evaluation, for DL and UL

·        Baseline: DL and UL performances are evaluated independently

·        Optional: DL and UL performance are evaluated together

·        FFS details both the baseline and the optional evaluations

Agreements: For Dense urban for XR/CG evaluation, update the agreement in RAN1 #103e for channel model as follows.

·        Dense urban: FR1 and FR2

o   Channel model: UMi UMa. Detailed definition of UMi UMa refers to TR 38.901.

Agreements: For XR/CG evaluation, adopt 12 degree for downtilt for Dense Urban in FR1.

·        Other downtilt value can also be optionally evaluated

Agreements: To facilitate further discussion on evaluation of power saving effect of different power saving schemes, the following references are defined.

• Case 1 (baseline): UE power consumption assuming UE is always ON, i.e., UE is always available for gNB scheduling.
• Case 2 (FFS optional or baseline): UE power consumption assuming Rel-15/16 CDRX configuration
• FFS CDRX configuration details
• Company can also optionally evaluate for other cases, e.g.
• Genie: UE power consumption assuming that UE is in a sleep state (e.g., micro/light/deep sleep as defined in TR38.840) whenever there is neither DL data reception nor UL transmission. From the gNB scheduling perspective, UE is always available for scheduling, i.e., there is no difference from Baseline in gNB scheduling and corresponding UE Tx/Rx. It is noted that Genie is not a power saving scheme but the result may serve as an upper bound of power saving gain of power saving techniques, which may potentially motivate development of new power saving techniques that can approach the Genie performance.
• R15/16/17 power saving techniques for connected mode, e.g., BWP, PDCCH skipping, search space switching, etc.

Decision: As per email posted on Feb 5^th,

Agreements:

UE power consumption (i.e., power saving gain of the evaluated scheme) for XR is evaluated in conjunction with impact on latency, user experience, and capacity.  In this regard, the following table is used to collect results for system level simulation from companies as a starting point.

·        FFS all UEs or only satisfied UEs are included for obtaining the PS gain

Table 1 Evaluation of UE power saving schemes for e.g., {dense urban, AR, FR1}

Note 1: CDF of power saving gains of each UE

Note 2: # of satisfied UEs per cell among # of UEs per cell (=N). 

Note 3: # of dropped UEs per cell (=N) that needs to be the same for all power saving schemes to be evaluated.

Note 4: company to provide the detailed simulation assumptions including parameter values for each case, e.g. CDRX parameters

Note 5: company can report one or more power saving gain metrics (i.e. mean PS gain or PS gain of 5%/50%/95%/-tile UE in PSG CDF) for each power saving scheme

Agreements: For UL UE power consumption evaluation for UE with transmit power X [0,23] dBm, adopt the following

• Option 1 (Baseline): Consider only two Tx power values as defined in TR 38.840
• Power number is given as A for X= [0, M)dBm and B for X =[M, 23]dBm, where A and B (defined in 38.840) correspond to power consumption numbers for a given uplink slot for 0dBm and 23dBm respectively.
• M = [20]
• Other value(s) of M can be optionally evaluated
• Companies to provide detailed assumptions on UE power consumption for Tx power values other than 0 and 23 dBm
• E.g. Power number is given as A for X= [0, 20)dBm and B for X =[20, 23]dBm, where A and B (defined in 38.840) correspond to power consumption numbers for a given uplink slot for 0dBm and 23dBm respectively.
• Option 2 (FFS mandatory or optional): Linear interpolation method in linear scale for Tx power values other than 0 dBm and 23 dBm

·        FFS whether or not to differentiate the two options (e.g., mandatory vs. optional)

·        FFS whether or not to consider UE with transmit power less than 0 dBm

8.14.3     Initial Performance Evaluation Results

Placeholder only – no email/GTW discussion

R1-2100363         Evaluation results of XR performance           CATT

R1-2100478         Initial performance evaluation results of XR vivo

R1-2100530         Preliminary Performance Evaluation Results for XR  ZTE , Sanechips

R1-2100587         Initial Performance and Evaluation Results for XR and CG      MediaTek Inc.

R1-2100682         Initial results for XR          Intel Corporation

R1-2100726         Initial Performance Evaluation Results          Nokia, Nokia Shanghai Bell

R1-2101068         Discussion on XR evaluation           CMCC

R1-2101255         Initial evaluation results for XR and Cloud Gaming    Huawei, HiSilicon

R1-2101316         Initial XR performance evaluation results     Ericsson

R1-2101495         Initial XR Performance Evaluation Results   Qualcomm Incorporated

8.14.44     Other

R1-2100134         Discussion on the  support of XR/CG service in sidelink-unlicensed      OPPO

R1-2100364         Potential area of NR enhancement for the support of XR services           CATT

R1-2100479         Challenges and potential enhancements of XR            vivo

R1-2100531         Considerations on XR specific enhancements              ZTE , Sanechips

R1-2100573         Discussion on potential enhancements for supporting XR         InterDigital, Inc.

R1-2101103         Discussion on potential enhancement for supporting XR           Xiaomi

R1-2101269         Other aspects for XR and Cloud Gaming      Huawei, HiSilicon

R1-2101317         Mobility and XR applications          Ericsson

R1-2101367         Views on potential enhancements for XR      Apple

 RAN1#104-bis-e

8.14   Study on XR Evaluations for NR (RAN1)

Please refer to RP-201145 for detailed scope of the WI

R1-2103988        Session notes for 8.14 (Study on XR Evaluations for NR)    Ad-Hoc Chair (Ericsson)

R1-2103190         Updated Work Plan for Rel-17 SI on XR Evaluations for NR   Qualcomm Incorporated

//Handled under NWM – See RAN1-104b-e-NWM-NR-XR-04 as the document name

[104b-e-NR-XR-04] – Eddy (Qualcomm)

Email discussion/approval whether/how to reply LS R1-2102308 till 4/16.

R1-2104117        LS response on New Standardized 5QIs for 5G-AIS (Advanced Interactive Services)             RAN1, Qualcomm Incorporated

Decision: As per email decision posted on April 20^th, the LS is approved.

8.14.1     Traffic Model

R1-2102320         Traffic model for XR and Cloud Gaming      Huawei, HiSilicon

R1-2102418         Discussion on the XR traffic models for evaluation    OPPO

R1-2102546         Discussion on traffic models of XR vivo

R1-2102616         XR traffic model CATT

R1-2102686         Traffic Model for XR and CG          MediaTek Inc.

R1-2102769         XR traffic model FUTUREWEI

R1-2102827         On Traffic Model for XR study       Nokia, Nokia Shanghai Bell

R1-2102955         Traffic model for XR         Ericsson

R1-2102969         Discussion on Traffic Model for XR services              Xiaomi

R1-2103054         Traffic Model for XR        Intel Corporation

R1-2103128         Views on XR traffic model              Apple

R1-2103192         Remaining Issues on XR Traffic Models       Qualcomm Incorporated

R1-2103264         Traffic model for XR         Samsung

R1-2103278         Further Discussion on Traffic Model for XR Evaluations         ZTE, Sanechips

R1-2103317         Considerations on XR traffic model Sony

R1-2103360         Discussion on traffic models for XR evaluation          LG Electronics

R1-2103429         UL traffic flows for XR applications             InterDigital, Inc.

R1-2103437         XR Traffic Model Considerations   AT&T

R1-2103598         Discussion on traffic model for XR NTT DOCOMO, INC.

[104b-e-NR-XR-01] – Eddy (Qualcomm)

Email discussion/approval on traffic model with checkpoints for agreements on Apr-15, Apr-20

R1-2104116        FL Summary of 104bis-e agreements and email discussions on XR traffic models               Moderator (Qualcomm)

Agreement:

Jitter for DL video stream for the case of a single stream per UE 

• J is drawn from a truncated Gaussian distribution:

o   Mean: 0 ms

o   STD: 2 ms

o   Range: [-4, 4] ms (baseline), [-5, 5] ms (optional)

§  Note: The values are set to ensure that packet arrivals are in order (i.e., arrival time of next packet is always larger than that of the previous packet) rather than the real measurement

o   Other values can be optionally evaluated

• Note: The above parameters for random variable J are effectively identical to the following parameter values because air interface PDB (e.g., 10ms or 15ms) applies from the point when each packet arrives at gNB as agreed in RAN1#104-e.

o   Mean: 4 ms (baseline), 5ms (optional)

o   STD: 2 ms

o   Range: [0, 8] ms (baseline), [0, 10] ms (optional)

o   Other values can be optionally evaluated

Agreement:

Parameters of Truncated Gaussian distribution for packet size of DL video stream in case of single stream evaluation (note: these parameter values are those before the truncation):

·        [STD, Max, Min]: [10.5, 150, 50]% of Mean packet size

·        Other values that can be used for evaluation: [STD, Max, Min] = [4, 112, 88] % of Mean for single eye buffer, [3, 109, 91] % of Mean for dual eye buffer

·        FFS: Whether and how to evaluate single eye and dual eye buffer

·        Note: Companies report the values used in their simulation results.

·        Note: There is no consensus that the [10.5, 150, 50]% of mean packet size is the best set of parameters

Agreement:

In case of single stream per UE in DL, a UE is declared a satisfied UE if more than X (%) of packets are successfully delivered within a given air interface PDB.

• The baseline X value is 99.
• Other values of X can be optionally evaluated, e.g., X < = 95, X=99.9.
• Additional combinations of (X, PDB) values can be optionally evaluated, e.g.,
• (99, 7), (95, 13) for VR/AR
• (99, 12), (95, 18) for CG
• FFS: Different values for I-frame and P-frame if evaluation of them is agreed.

Agreement:

On UL Traffic model and QoS parameters

• CG/VR: single stream (pose/control)
• Traffic model for Pose/control

o   Periodic: 4ms (no jitter)

§  Other values can be optionally evaluated.

o   Fixed: 100 bytes

§  PDB: 10 ms.

o   A UE is declared a satisfied UE if more than X (%) of packets are successfully delivered within the given air interface PDB.

§  The baseline X value is 99.

§  Other X values can be optionally evaluated are 90 and 95.

Agreement:

In addition to single stream per UE in DL which is baseline, two streams can be optionally evaluated for DL

• Option 1: I-frame + P-frame
• Option 1A: slice-based traffic model
• Option 1B: Group-Of-Picture (GOP) based traffic model
• Option 2: video + audio/data
• Option 3: FOV + omnidirectional stream
• Companies should report detailed assumptions in their simulations on packet size distribution for each stream, packet arrival interval (or fps) for each stream, PDB for each stream, PER requirement for each stream, criteria for being satisfied.
• Companies should strive to align the parameter values for the options chosen as much as possible
• FFS: Whether audio stream is separate or aggregated with the data stream in option 2 (Intention of option 2 is not to create a 3 stream option)

Agreement:

For evaluations of AR in UL:

·        Option 1 (Baseline for power and capacity evaluations): Two streams as defined below

o   Stream 1: pose/control

§  Traffic model and QoS parameters are same as for pose/control for UL CG/VR.

o   Stream 2: A stream aggregating streams of scene, video, data, and audio.

§  Packet size: Truncated Gaussian distribution with the parameter values same as for DL

§  Periodicity: 60 fps

·        Jitter (optional): same model as for DL

§  Data rate: 10 Mbps (baseline), 20 Mbps (optional)

§  PDB: [60] ms (baseline), [10/15] ms (optional)

·        Option 2 (Optional for power evaluation and baseline for capacity evaluation): Single stream as defined below

o   Packet size: Truncated Gaussian distribution with the parameter values same as for DL

o   Periodicity: 60 fps

§  Jitter (optional): same model as for DL

o   Data rate: 10 Mbps (baseline), 20 Mbps (optional)

o   PDB: [60] ms (baseline), [10/15] ms (optional)

·        Option 3 (Optional): Three streams as defined below

o   Stream 1: pose/control

§  Traffic model and QoS parameters are same as for pose/control for UL CG/VR.

o   Stream 2: A stream aggregating streams of scene and video

§  Packet size: Truncated Gaussian distribution with the parameter values same as for DL

§  Periodicity: 60 fps

·        Jitter (optional): same model as for DL

§  Data rate: 10 Mbps (baseline), 20 Mbps (optional)

§  PDB: [60] ms (baseline), [10/15] ms (optional)

o   Stream 3: A stream aggregating streams of audio and data

§  Periodicity: 10ms

§  Data rate: 0.756 Mbps/s or 1.12 Mbps

§  Packet size: determined by periodicity and data rate

§  PDB: 30 ms

·        Option 4 (Optional): Three streams as defined below

o   Stream 1: pose/control

§  Traffic model and QoS parameters are same as for pose/control for UL CG/VR.

o   Stream 2: I-stream for video

o   Stream 3: P-stream for video

o   Note: For stream 2 and stream 3, the I/P-stream model for DL video can be reused for UL video.  Companies should report detailed assumptions in their simulations on packet size distribution for each stream, packet arrival interval (or fps) for each stream, PDB for each stream, PER requirement for each stream, criteria to be satisfied UE.

• Companies should strive to align the parameter values for the options chosen as much as possible

·        Note: Above PDB values in [ ] for Stream 2 in Option 1 and 3, and Option 2 are to be further discussed and potentially confirmed in RAN1#105-e, where other values can be also discussed if needed.

·        In case multiple steams are evaluated for UL AR, a UE is declared as satisfied only when each stream meets the requirement that X (%) of packets are successfully delivered within a given air interface PDB.

o   X value for pose/control: follow X values for pose/control for CG/VR

o   X value for other stream: follow X values for DL video stream.

8.14.2     Evaluation Methodology

Including identification of KPIs of interest for relevant deployment scenarios

R1-2102321         Evaluation methodology for XR and Cloud Gaming   Huawei, HiSilicon

R1-2102419         Discussion on the XR evaluation methodology           OPPO

R1-2102547         Discussion on evaluation methodologies of XR           vivo

R1-2102613         Evaluation methodology and performance index for XR           CATT

R1-2102687         On Evaluation Methodology for XR and CG MediaTek Inc.

R1-2102770         XR evaluation methodology            FUTUREWEI

R1-2102828         Development of the Evaluation Methodology for XR Study     Nokia, Nokia Shanghai Bell

R1-2102956         Evaluation methodology for XR      Ericsson

R1-2102970         Discussion on evaluation methodology for XR services            Xiaomi

R1-2103055         Evaluation Methodology   Intel Corporation

R1-2103129         Views on XR evaluatoin methodology          Apple

R1-2103193         Remaining Issues on Evaluation Methodology for XR              Qualcomm Incorporated

R1-2103265         Evaluation methodology and KPIs for XR     Samsung

R1-2103279         On XR Evaluation Methodology     ZTE, Sanechips

R1-2103361         Discussion on evaluation methodologies for XR         LG Electronics

R1-2103430         Remaining Issues on XR Evaluations and KPIs           InterDigital, Inc.

R1-2103438         XR Evaluation Assumptions            AT&T

R1-2103599         Discussion on evaluation methodology for XR            NTT DOCOMO, INC.

R1-2103827        Summary of email discussion for evaluation methodology and assumptions               Moderator (vivo)

[104b-e-NR-XR-02] – Xiaohang (Vivo)

Email discussion/approval on evaluation methodology with checkpoints for agreements on Apr-15, Apr-20

R1-2104127        Summary of [104b-e-NR-XR-02] Email discussion on evaluation methodology               Moderator (vivo)

R1-2104128         Draft template for XR evaluation results       Moderator (vivo)

Agreement:

·        Case 2, i.e. CDRX, is optionally evaluated for UE power consumption evaluation

Agreement:

·        For XR power consumption evaluation, CDRX parameters are reported by companies

Agreement:

For UL UE power consumption evaluation, the following is encouraged

• Linear interpolation method in linear scale for Tx power values other than 0 dBm and 23 dBm
• Companies should indicate how they do linear interpolation method in linear scale considering step-wise linear average of UE power model
• FFS: Further clarifications on linear interpolation method in linear scale considering step-wise linear average of UE power model
• Other methods that can be used for evaluation: Consider only two Tx power values as defined in TR 38.840
• Power number is given as A for X= [0, M]dBm and B for X =[M, 23]dBm, where A and B (defined in 38.840) correspond to power consumption numbers for a given uplink slot for 0dBm and 23dBm respectively.
• M = [20]
• Other value(s) of M can be optionally evaluated

From GTW session, confirmed by email posted on April 21^st,

Agreement:

For XR/CG capacity evaluation, when DL and UL performances are evaluated independently, the system capacity for DL capacity and UL capacity are reported respectively.

• FFS whether/how to determine the joint capacity for DL and UL after companies have submitted evaluation results

Conclusion:

It is up to companies to choose either Option 1 (DDDSU) or Option 2 (DDDUU) for TDD configuration (as per previous agreements) and do the evaluation.

Agreement:

It is up to each company to report the following performance metrics optionally

• Percentage of satisfied UEs
• CDF of packet error ratio
• CDF of packet latency
• CDF of user-perceived throughput
• Resource utilization

Note: it does not mean all the optional performance metrics will be captured in the TR. How to use these optional reported metrics and whether to capture in the TR can be separate discussion after there are substantial evaluation results.

Agreement: 

For XR power evaluation (including baseline and power saving schemes), companies report both Option 1 and Option 2 results for evaluating the power saving gain.

• Option 1: all UEs are considered
• Option 2: satisfied UEs only are considered

Agreement:

For XR/CG power consumption evaluation, for DL and UL,

• Option 1: DL and UL performances are evaluated independently. DL and UL power consumption results are collected separately.
• Option 3: DL and UL performances are evaluated together. DL and UL power consumption are counted to obtain the total power consumption
• Companies to report the assumptions for power consumption evaluation

Agreement:

For XR UE power consumption evaluation

• The same number of UE per cell are used in baseline and power saving schemes,
• Note: the number of satisfied UEs is reported in the power evaluations (already agreed in RAN1 #104-e).
• Max users/cell at which UE can meet the capacity KPI should be reported for baseline and for different UE PS techniques.
• Results for other cases (e.g. power savings gain for lightly loaded case) can also be reported optionally.
• The system capacity for each case (e.g. a given number of UE per cell) for evaluating power saving schemes is reported in power evaluation

Conclusion:

It is up to company to report either equal number of UEs per cell or unequal number of UEs per cell is assumed for capacity evaluation.

• Note: unequal number of UEs per cell means even average load per cell.

Agreement:

For XR/CG capacity evaluation, a packet is considered as lost when it has exceeded the PDB, such that it will be added to the PER and the data of the packet is discarded.

• It is up to company to report the details for the packet when it has exceeded the PDB, e.g.
• Option 1: The packet exceeding the delay is still delivered to the other side
• Option 2: The packet (including the non-transmitted part) is discarded at the transmitter (at the gNB for DL packets and at the UE for UL packets)
• Other options are not precluded
• Note: This is for the purpose of evaluation

8.14.3     Initial Performance Evaluation Results

R1-2102322         Initial evaluation results for XR and Cloud Gaming    Huawei, HiSilicon

R1-2102420         Initial performance results for XR evaluation              OPPO

R1-2102548         Initial performance evaluation results of XR vivo

R1-2102614         Evaluation results of XR performance           CATT

R1-2102707         8.14.3  Initial Performance and Evaluation Results for XR and CG         MediaTek Inc.

R1-2102771         XR initial evaluations        FUTUREWEI

R1-2102829         Performance results in indoor hotspot and dense urban deployments of CG and VR/AR applications          Nokia, Nokia Shanghai Bell

R1-2102904         Initial XR Evaluation Results          CMCC

R1-2102957         Initial XR performance evaluation results     Ericsson

R1-2103833         Initial performance evaluation on XR            Apple    (rev of R1-2103130)

R1-2103194         Initial Evaluation Results for XR Capacity and UE Power Consumption Qualcomm Incorporated

R1-2103280         Initial Performance Evaluation Results for XR            ZTE, Sanechips

R1-2103431         Performance Evaluation Results on XR applications  InterDigital, Inc.

R1-2103439         XR Initial Performance Results       AT&T

[104b-e-NR-XR-03] – Eddy (Qualcomm)

Email discussion/approval on initial performance evaluation results with checkpoints for agreements on Apr-15, Apr-20

R1-2104118        FL Summary of initial evaluation results for XR over NR   Moderator (Qualcomm)

Decision: The document is for information. As per email posted on April 21^st, the discussion on initial evaluation results provides useful information for the next round of evaluations to be captured in the TR.

8.14.44     Other

R1-2102421         Discussion on the  support of XR/CG service in sidelink-unlicensed      OPPO

R1-2102549         Challenges and potential enhancements of XR            vivo

R1-2102615         Potential area of NR enhancement for the support of XR services           CATT

R1-2102830         Potential enhancements for better XR support over NR             Nokia, Nokia Shanghai Bell

R1-2102958         Mobility and XR applications          Ericsson

R1-2102971         Discussion on potential enhancement for supporting XR           Xiaomi

R1-2103131         Views on potential enhancements for XR      Apple

R1-2103195         Potential Enhancements for XR       Qualcomm Incorporated

R1-2103281         Considerations on XR Specific Enhancement              ZTE, Sanechips

R1-2103390         Challenges and potential enhancements for XR and Cloud Gaming        Huawei, HiSilicon

R1-2103432         Potential enhancements for supporting XR    InterDigital, Inc.

 RAN1#105-e

8.14   Study on XR Evaluations for NR (RAN1)

Please refer to RP-201145 for detailed scope of the WI

R1-2106238        Session notes for 8.14 (Study on XR Evaluations for NR)    Ad-Hoc Chair (Samsung)

R1-2104700         Updated Work Plan for Rel-17 SI on XR Evaluations for NR   Qualcomm Incorporated

8.14.1     Traffic Model

R1-2104207         XR traffic model FUTUREWEI

R1-2104238         Traffic model for XR and Cloud Gaming      Huawei, HiSilicon

R1-2104395         Remaining issues on traffic models of XR    vivo

R1-2104502         XR traffic model CATT

R1-2104555         On Traffic Model for XR study       Nokia, Nokia Shanghai Bell

R1-2104701         Remaining Issues on XR Traffic Models       Qualcomm Incorporated

R1-2104745         Discussion on the XR traffic models for evaluation    OPPO

R1-2104934         Traffic Model for XR        Intel Corporation

R1-2105134         Considerartions on XR traffic model             Apple

R1-2105181         Considerations on XR traffic model Sony

R1-2105342         Traffic Models for XR       Samsung

R1-2105376         Traffic Model for XR and CG          MediaTek Inc.

R1-2105443         Discussion on traffic models for XR evaluation          LG Electronics

R1-2105499         Discussion on UL traffic models     InterDigital, Inc.

R1-2105547         Discussion on remaining issues of traffic Model for XR services            Xiaomi

R1-2105603         Remaining Issues of XR Traffic Model         ZTE, Sanechips

R1-2105726         Discussion on traffic model for XR NTT DOCOMO, INC.

R1-2105829         Traffic model for XR         Ericsson

From AI 5 (Related to R1-2102308 Reply LS on New Standardized 5QIs for 5G-AIS (Advanced Interactive Services)          SA4, Qualcomm)

R1-2104749         Draft reply LS on New Standardized 5QIs for 5G-AIS (Advanced Interactive Services)              OPPO

R1-2105948         [Draft] LS response on New Standardized 5QIs for 5G-AIS     Qualcomm Inc.

[105-e-NR-XR-01] – Eddy (Qualcomm)

Email discussion/approval on traffic model

·        Including discussion and possible reply LS for R1-2104023 and R1-2102308

·        1^st check point: May 24

·        2^nd check point: May 27

R1-2106380        [104b-e-NR-XR-01] Email discussion/approval on traffic model       Moderator (Qualcomm)

From GTW sessions:

Agreement

In addition to the response LS from RAN1#104-bis-e in April 2021 to SA2 and SA4 (cc: RAN2) in R1-2104117, RAN1 would like to provide the following information in response to the LS from SA4, based on additional evaluation results: even though RAN1 hasn’t performed evaluations with the exact parameters (e.g. in RAN1 evaluations, data rate higher than 45Mbps was not considered and simulation was frame based) presented by SA4 (5QIs), it is RAN1 understanding that these values can be supported by NG-RAN.

R1-2106149        LS response on New Standardized 5QIs for 5G-AIS (Advanced Interactive Services)             RAN1, Qualcomm

Decision: The LS is approved. MCC to correct sourcing to RAN1 only.

Agreement

PDB value of the stream in UL AR aggregating streams of scene, video, data, and audio, i.e., Option 2, Stream 2 in Option 1, and Stream 2 in Option 3.

·        30ms (baseline), 10/15/60ms (optional)

Agreement

For DL video stream, separate packet arrivals in time for dual-eye buffer can be optionally evaluated, based on the single stream model by doubling the packet arrival rate and halving the packet size compared to the single stream, while all other parameters (e.g., jitter, PDB) are the same as for single stream. 

·        For companies who are evaluating separate packet arrivals in time for dual-eye buffer in addition to single stream (baseline), it is recommended to evaluate at least the following scenarios in the table.  It is encouraged to evaluate additional baseline/optional scenarios/configurations.

Agreement

When companies are submitting evaluation results to RAN1, it is recommended to submit results at least the following parameters in the below table.

·        Note 1: This is only intended to have more results from more companies at least for the corresponding configuration. RAN1 agreements regarding baseline vs. optional for simulation scenarios, configurations, parameters, remain the same. 

·        Note 2: Companies are encouraged to submit results for other baseline/optional configurations as much as they can.

Agreement

For the optional evaluation scenario, two streams of I-frame and P-frame for DL video stream (option 1), the traffic models described in the below table are assumed.

·      FFS: Parameter values of , A, B, C, D, E, F, G, H

o   Including the possibility of using multiple set of parameter values

·        For companies who are evaluating this option, it is recommended to evaluate at least the following scenario: AR/VR, 30Mbps, Dense Urban for FR1 and InH for FR2.  It is encouraged to evaluate additional baseline/optional scenarios/configurations.

8.14.2     Evaluation Methodology

Including identification of KPIs of interest for relevant deployment scenarios

R1-2104208         XR evaluation methodology            FUTUREWEI

R1-2104239         Evaluation methodology for XR and Cloud Gaming   Huawei, HiSilicon

R1-2104396         Discussion on evaluation methodologies for XR         vivo

R1-2104499         Evaluation methodology and performance index for XR           CATT

R1-2104556         Development of the Evaluation Methodology for XR Study     Nokia, Nokia Shanghai Bell

R1-2104702         Remaining Issues on Evaluation Methodology for XR              Qualcomm Incorporated

R1-2104746         Discussion on the XR evaluation methodology           OPPO

R1-2104935         Evaluation Methodology for XR     Intel Corporation

R1-2105135         Remaining issues in XR evaluation methodology       Apple

R1-2105343         Evaluation methodology and KPIs for XR     Samsung

R1-2105377         On Evaluation Methodology for XR and CG MediaTek Inc.

R1-2105444         Discussion on evaluation methodologies for XR         LG Electronics

R1-2105500         Discussion on additional issues on XR Evaluations Methodology and KPI               InterDigital, Inc.

R1-2105548         Discussion on remaining issues of evaluation methodology for XR services               Xiaomi

R1-2105604         Further Discussion on XR Evaluation Methodology   ZTE, Sanechips

R1-2105727         Discussion on evaluation methodology for XR            NTT DOCOMO, INC.

R1-2105830         Evaluation methodology for XR      Ericsson

//Handled under NWM – See RAN1-105-e-NWM-NR-XR-02 as the document name

[105-e-NR-XR-02] – Xiaohang (vivo)

Email discussion/approval on evaluation methodology

-        1^st check point: May 24

-        2^nd check point: May 27

R1-2105997        Summary #1 of evaluation methodology for XR     Moderator (vivo)

From GTW session:

Agreement

Confirm the 2-symbol gap at the end to third “D” slot of DDDUU for FR1/FR2.

·        Applies only for Option 2

Agreement

UE with transmit power less than 0 dBm is considered for power consumption evaluation, adopt option 2 as baseline, i.e. the power model of 0 dBm for UE with transmit power less than 0 dBm.

·        Option 1 can be optionally evaluated

·        Note: Above is not intended to introduce new power class

R1-2106096        Summary#2 of evaluation methodology for XR      Moderator (vivo)

R1-2106172        Summary#3 of evaluation methodology for XR      Moderator (vivo)

From GTW session:

Agreement

For FR2, it is up to company to report the UE UL power consumption model.

R1-2106283        Summary #4 of evaluation methodology for XR     Moderator (vivo)

From GTW session:

For companies to further study and if necessary, discuss in RAN1#106-e

(Coverage evaluation methodology) For XR/CG in DL or UL, coverage is defined to be the A-percentile point in CDF of Coupling gain for the “satisfied” UEs, with #UEs per cell = B, for a given XR application (AR/VR/CG) in a given deployment scenario (DU/InH/UMa)

·        A = [5], other value can also be reported

·        FFS: Value of B, e.g. B = 1, capacity, etc.

·        Note: Coupling gain for coverage evaluation is defined as the ratio of received and transmitted power measured in dB, and includes antenna gains, path loss, shadowing, indoor- or body loss, etc. Example of coupling gain can refer to TR 37.910.

An alternate method could be to use the “traditional” method such as what is used in the CE study/work item.

Final summary and updated template in:

R1-2106371        Summary of [105-e-NR-XR-02] Email discussion/approval on evaluation methodology      Moderator (vivo)

R1-2106372        Updated template for XR evaluation results            Moderator (vivo)

8.14.3     Initial Performance Evaluation Results

R1-2104209         XR initial evaluations        FUTUREWEI

R1-2104397         Initial performance evaluation results on XR vivo

R1-2104500         Evaluation results of XR performance           CATT

R1-2104557         Performance results in indoor hotspot and dense urban deployments of CG and VR applications         Nokia, Nokia Shanghai Bell

R1-2104703         Initial Evaluation Results for XR Capacity and UE Power Consumption Qualcomm Incorporated

R1-2104747         Evaluation results for XR evaluation             OPPO

R1-2105963         Initial results for XR          Intel Corporation (rev of R1-2104936)

R1-2105136         Performance evaluation on XR        Apple

R1-2105392         Initial Performance and Evaluation Results for XR and CG      MediaTek Inc.

R1-2106010         Initial Performance Evaluation Results on XR             InterDigital, Inc.  (rev of R1-2105501)

R1-2105521         Initial evaluation results for XR and Cloud Gaming    Huawei, HiSilicon

R1-2105605         Performance Evaluation Results for XR        ZTE, Sanechips

R1-2106057         XR Initial Performance Results       AT&T    (rev of R1-2105664)

R1-2105831         Initial XR performance evaluation results     Ericsson

[105-e-NR-XR-03] – Eddy (Qualcomm)

Email discussion/approval on initial performance evaluation results

-        1^st check point: May 24

-        2^nd check point: May 27

R1-2106116        Evaluation Results for XR Capacity and UE Power Consumption    Moderator (Qualcomm)

Decision: No conclusion made or agreements endorsed. Only key observations are provided in section 2 of x6116.

8.14.44     Other

R1-2104398         Challenges and potential enhancements of XR            vivo

R1-2104501         Potential area of NR enhancement for the support of XR services           CATT

R1-2104558         Enhancements for better XR support over NR             Nokia, Nokia Shanghai Bell

R1-2104704         Potential Enhancements for XR       Qualcomm Incorporated

R1-2104748         Discussion on the  support of XR/CG service in sidelink-unlicensed      OPPO

R1-2105137         Considerations on potential enhancements for XR      Apple

R1-2105344         Potential enhancements for better XR support in NR  Samsung

R1-2105502         Discussion on potential enhancements for XR             InterDigital, Inc.

R1-2105522         Challenges and potential enhancements for XR and Cloud Gaming        Huawei, HiSilicon

R1-2105549         Discussion on potential enhancement for supporting XR           Xiaomi

R1-2105606         Further Discussion on Capacity and Power Working Areas for XR         ZTE, Sanechips

R1-2105832         Discussion on enhancements for XR              Ericsson

R1-2106038        Summary of AI 8.14.4 on potential enhancements for XR   Moderator (vivo)

 RAN1#106-e

8.14   Study on XR Evaluations for NR (RAN1)

Please refer to RP-210460 for detailed scope of the WI

R1-2108612        Session notes for 8.14 (Study on XR Evaluations for NR)    Ad-Hoc Chair (CMCC)

8.14.1     Traffic Model

R1-2106456         Traffic model for XR and Cloud Gaming      Huawei, HiSilicon

R1-2106526         Remaining Issues of XR Traffic Model         ZTE, Sanechips

R1-2106629         Remaining issues on traffic models of XR    vivo

R1-2106917         Traffic Models for XR       Samsung

R1-2106949         XR traffic model CATT

R1-2107131         Discussion on Traffic Model for XR/CG       China Telecom

R1-2108213         Discussion on the XR traffic models for evaluation    OPPO     (rev of R1-2107279)

R1-2107374         Remaining Issues on XR Traffic Models       Qualcomm Incorporated

R1-2107461         Discussion on traffic models for XR evaluation          LG Electronics

R1-2107500         Traffic Model for XR and CG          MediaTek Inc.

R1-2107534         Discussion on UL traffic models for AR       InterDigital, Inc.

R1-2107616         Traffic model for XR         Intel Corporation

R1-2107629         Traffic model for XR         Ericsson

R1-2107768         Remaining issues in XR traffic model           Apple

R1-2107886         Discussion on traffic model for XR NTT DOCOMO, INC.

R1-2107905         Discussion on remaining issues of traffic Model for XR services            Xiaomi

[106-e-NR-XR-01] – Eddy (Qualcomm)

Email discussion/approval on traffic model

• Including discussion and possible reply LS for R1-2104023
• 1^st check point: August 19
• 2^nd check point: August 24
• Final check: August 27

R1-2108500        FL proposals for remaining issues of XR traffic model        Moderator (Qualcomm)

From GTW session:

Agreement

·        For DL multi-stream evaluations, a UE is declared as a satisfied UE if each stream meets the PER and PDB requirements of that stream, i.e., more than a certain percentage of packets are successfully transmitted within a given air interface PDB.

Agreement

For Option 2 (video + audio/data) of evaluation of DL two streams that is an optional evaluation scenario, the audio/data flow is modelled as follows:

·        A stream aggregating streams of audio and data

o   Periodicity: 10ms

o   Data rate: 0.756 Mbps/s or 1.12 Mbps

o   Packet size: determined by periodicity and data rate

o   PDB: 30ms (baseline).  Other values can be optionally evaluated.

o   PER: 1% (baseline). Other values, e.g., 0.1%, can be optionally evaluated.

Agreement

For evaluation of separate streams of I-frame and P-frame that is an optional evaluation scenario,

• The main objective of evaluating this option is to study the impact on capacity from different PDB and PER values for I-frame and P-frame. 
• FFS: Whether to directly compare capacity results (i.e., capacity numbers) for cases with two-stream modelling and those for cases with single-stream modelling.

Agreement

For evaluation of separate streams of I-frame and P-frame that is an optional evaluation scenario,

·        Alpha value: 2.0 and 1.5, Other values, e.g., 3.0 can be optionally evaluated

o   This alpha value assumption applies to both Option 1A (slice-based) and Option 1B (GOP-based) evaluations

Agreement

For evaluation of separate streams of I-frame and P-frame that is an optional evaluation scenario,

·        RAN1 agree upon the below reference case, while leaving other study cases up to companies.

o   Reference case

• For DL
• [PER_I, PER_P, PDB_I, PDB_P] = [1 %, 1 %, 10ms, 10ms] for AR/VR
• [PER_I, PER_P, PDB_I, PDB_P] = [1 %, 1 %, 15ms, 15ms] for CG
• For UL AR video streams
• [PER_I, PER_P, PDB_I, PDB_P] = [1 %, 1 %, 30ms, 30ms]

8.14.2     Evaluation Methodology

Including identification of KPIs of interest for relevant deployment scenarios

R1-2106457         Evaluation methodology for XR and Cloud Gaming   Huawei, HiSilicon

R1-2106527         Remaining Issues of XR Evaluation Methodology      ZTE, Sanechips

R1-2106630         Remaining issues on evaluation methodologies for XR             vivo

R1-2106918         Evaluation methodology and KPIs for XR     Samsung

R1-2106950         Evaluation methodology and performance index for XR           CATT

R1-2107087         XR evaluation methodology            FUTUREWEI

R1-2107280         Discussion on the XR evaluation methodology           OPPO

R1-2107375         Remaining Issues on Evaluation Methodology for XR              Qualcomm Incorporated

R1-2107462         Discussion on evaluation methodologies for XR         LG Electronics

R1-2107501         On Evaluation Methodology for XR and CG MediaTek Inc.

R1-2107535         Remaining Issues on XR Evaluation Methodologies   InterDigital, Inc.

R1-2107617         Evaluation Methodology for XR     Intel Corporation

R1-2107630         Evaluation methodology for XR      Ericsson

R1-2107656         Development of the Evaluation Methodology for XR Study     Nokia, Nokia Shanghai Bell

R1-2107769         Considerations on XR evaluation methodology           Apple

R1-2107887         Discussion on evaluation methodology for XR            NTT DOCOMO, INC.

R1-2107906         Discussion on remaining issues of evaluation methodology for XR services               Xiaomi

[106-e-NR-XR-02] – Eddy (Qualcomm)

Email discussion/approval on evaluation methodology

• 1st check point: August 19
• 2nd check point: August 24
• Final check: August 27

R1-2108501        FL proposals for remaining issues of evaluation methodology for XR               Moderator (Qualcomm)

From GTW session:

Agreement

Optional methodology 1 for XR coverage evaluation

·        For XR/CG in DL or UL, coverage is defined to be the A-percentile point in CDF of coupling gain for the “satisfied” UEs, with #UEs per cell = B, for a given XR application (AR/VR/CG) in a given deployment scenario (DU/InH/UMa)

o   A = 5

o   B = 1 and/or capacity

·        Coupling gain for coverage evaluation is defined as the ratio of received and transmitted power measured in dB, and includes antenna gains, path loss, shadowing, indoor- or body loss, etc. Example of coupling gain can refer to TR 37.910.

·        Note: The evaluation of coupling gain will be impacted by e.g., interference and scheduler mechanism, etc.

Optional methodology 2 for XR coverage evaluation

·        For each drop,

o   Randomly drop only one UE in the entire network (or in all the cells) that is associated with one of the 3 center cells (or gNBs), i.e., only one of the center gNBs is activated. 

o   Coupling gain for coverage evaluation is defined as the ratio of received and transmitted power measured in dB, and includes antenna gains, path loss, shadowing, indoor- or body loss, etc. Example of coupling gain can refer to TR 37.910.

o   Run SLS according to capacity evaluation methodology and determine whether the UE is satisfied or not.

·        Definition of the XR coverage

o   X %-tile point in the CDF curve of coupling gain for all the satisfied UEs, where X = 5.

Note: It will be further discussed how to capture the result in the TR.

8.14.3     Initial Performance Evaluation Results

R1-2108273         Performance Evaluation Results for XR        ZTE, Sanechips   (rev of R1-2108209, rev of R1-2106528)

R1-2106631         Performance evaluation results for XR          vivo

R1-2106951         Evaluation results of XR performance           CATT

R1-2107088         XR initial evaluations        FUTUREWEI

R1-2108377         Evaluation results for XR evaluation             OPPO     (rev of R1-2107281)

R1-2108251         Evaluation Results for XR Capacity and Power           Qualcomm           (rev of R1-2107376)

R1-2107429         Initial XR Evaluation Results          CMCC

R1-2107502         Initial Performance and Evaluation Results for XR and CG      MediaTek Inc.

R1-2107536         Performance Evaluation Results for XR        InterDigital, Inc.

R1-2108239         Initial results for XR          Intel Corporation (rev of R1-2107618)

R1-2107657         Performance results in indoor hotspot and dense urban deployments of CG and VR/AR applications          Nokia, Nokia Shanghai Bell

R1-2107666         Initial evaluation results for XR and Cloud Gaming    Huawei, HiSilicon

R1-2107694         XR Initial Performance Results       AT&T

R1-2107770         Performance evaluation on XR        Apple

R1-2107907         Initial performance evaluation result for XR Xiaomi

R1-2108488         XR performance evaluation results Ericsson (rev of R1-2108007)

R1-2108100         Initial evaluation results for XR       China Unicom

R1-2108202         Initial Performance and Evaluation Results for XR and CG      MediaTek Inc.

[106-e-NR-XR-03] Email discussion/approval on initial performance evaluation results – Xiaohang (vivo)

• 1st check point: August 19
• 2nd check point: August 24
• Final check: August 27

R1-2108314         Summary #1 of initial performance evaluation results for XR   Moderator (vivo)

R1-2108664        Summary of [106-e-NR-XR-03] email discussion on XR evaluation results               Moderator (vivo)

From GTW session:

Agreement

·        For capturing the observations for capacity for XR, following aspects are considered.

o   Baseline capacity performance

o   Various parameters/modeling on capacity performance, including the capacity performance with different assumptions, capacity performance with multi-stream traffic model, etc.

o   FFS: Potential enhancement on capacity performance

·        For capturing the observations for various parameters/modeling on capacity performance, the following could be considered

o   Data-rate on capacity

o   PDB/PER on capacity

o   Multi-stream traffic on capacity

o   Jitter impact on capacity

o   Etc.

·        FFS: For capturing the observations from the evaluation results for potential enhancement on capacity performance

R1-2108665        Summary of XR evaluation results_RAN1-106e     Moderator (vivo)

8.14.44     Other

R1-2106529         On Capacity and Power Working Areas for XR           ZTE, Sanechips

R1-2106632         Challenges and potential enhancements for XR           vivo

R1-2106822         Considerations on potential enhancements for XR      Sony

R1-2106919         Potential enhancements for XR       Samsung

R1-2106952         Potential area of NR enhancement for the support of XR services           CATT

R1-2107158         Potential enhancements of NR to support XR services              NEC

R1-2107282         Discussion on the  support of XR/CG service              OPPO

R1-2107377         Potential Enhancements for XR       Qualcomm Incorporated

R1-2107537         Discussion on potential enhancements for XR             InterDigital, Inc.

R1-2107631         Discussion on enhancements for XR              Ericsson

R1-2107658         Enhancements for better XR support over NR             Nokia, Nokia Shanghai Bell

R1-2107667         Challenges and potential enhancements for XR and Cloud Gaming        Huawei, HiSilicon

R1-2107771         Potential enhancements for XR       Apple

R1-2107908         Discussion on potential enhancement for supporting XR           Xiaomi

 RAN1#106-bis-e

8.14   Study on XR Evaluations for NR (RAN1)

Please refer to RP-201145 for detailed scope of the WI.

R1-2110702        Session notes for 8.14 (Study on XR Evaluations for NR)    Ad-Hoc Chair (CMCC)              (rev of R1-2110621)

R1-2110215         TR for Study on XR Evaluations for NR       Qualcomm Incorporated

[106bis-e-NR-XR-03] – Eddy (Qualcomm)

Email discussion/approval on an updated version of TR 38.838

- 1^st check point: October 14

- Final check point: October 19

R1-2110677        TR for Study on XR Evaluations for NR   Rapporteur (Qualcomm)

Decision: As per email decision posted on Oct 22^nd, the draft TR update is endorsed in principle. All changes to be accepted and endorsed as v0.1.0 of TR38.838 in:

R1-2110678        TR38.838 v0.1.0 Study on XR (Extended Reality) evaluations for NR               Rapporteur (Qualcomm)

Note:V0.1.0 is to be used  as basis for future update.

8.14.1     Performance Evaluation Results

R1-2108736         Performance evaluation results for XR and Cloud Gaming       Huawei, HiSilicon

R1-2108799         XR evaluation results        FUTUREWEI

R1-2108869         Initial evaluation results for XR       CEWiT

R1-2108889         Performance Evaluation Results for XR        ZTE, Sanechips

R1-2109008         Performance evaluation results on XR           vivo

R1-2109100         Evaluation results for XR evaluation             OPPO

R1-2109200         Evaluation results of XR performance           CATT

R1-2109307         Performance evaluation results for XR          CMCC

R1-2109393         Performance evaluation result for XR            Xiaomi

R1-2109555         Performance Evaluation Results      MediaTek Inc.

R1-2110401         Performance evaluation results for XR and CG           Intel Corporation (rev of R1-2109638)

R1-2110386         Performance results in indoor hotspot and dense urban deployments of CG and VR/AR applications          Nokia, Nokia Shanghai Bell            (rev of R1-2109737)

R1-2109922         XR Performance Evaluation Results              AT&T

R1-2110394         Performance Evaluation Results for XR        InterDigital, Inc.  (rev of R1-2109924)

R1-2110403         XR performance evaluation results Ericsson (rev of R1-2110144)

R1-2110402         Evaluation Results for XR Capacity, UE Power, and Coverage Qualcomm Incorporated        (rev of R1-2110216)

R1-2110246         Initial performance evaluation results of XR ITRI

R1-2110486         [DRAFT] Observations for XR capacity evaluations in TR       Moderators (Qualcomm, vivo)

[106bis-e-NR-XR-01] – Eddy (Qualcomm)

Email discussion/approval on XR performance evaluation results

-        1^st check point: October 14

-        Final check point: October 19

From Oct 12^th GTW session

For capacity enhancement

·        How to capture the results from multiple sources? e.g., in Section 2.1, range of result is quite big, which results if any should be taken out? (some filtering mechanism)

Note: How to handle some extreme value/results

·        Separate section for parameters and enhancements

For structure

·        Where and/or How to draw conclusion?

·        Source specific observation

·        All raw data will be captured

Note: Companies are expected to check whether their results & assumptions are captured correctly in the Appendix

Decision: As per email decision posted on Oct 22^nd, the following contributions capture the latest updates and are just noted to resume the discussion in the next meeting

R1-2110682        [DRAFT] Observations for XR capacity evaluations in TR Rapporteur (Qualcomm)

R1-2110683        [DRAFT] Observations for XR power evaluations in TR     Rapporteur (Qualcomm)

R1-2110684        [DRAFT] Observations for XR coverage evaluations in TR Rapporteur (Qualcomm)

8.14.2     Any Remaining Issues on Evaluation Methodology

Including any remaining details on traffic model, KPIs of interest for relevant deployment scenarios, etc.

R1-2108735         Traffic model and evaluation methodology for XR and Cloud Gaming  Huawei, HiSilicon

R1-2108890         Remaining Issues on XR Traffic and Evaluation Methodology ZTE, Sanechips

R1-2109009         Remaining issues on traffic model and EVM for XR  vivo

R1-2109101         Remaining issues on XR traffic model and evaluation methodology       OPPO

R1-2109111         Remaining issues on evaluation methodology             Ericsson

R1-2109198         Remaining issues of XR Evaluation methodology       CATT

R1-2109394         Discussion on remaining issues of evaluation methodology for XR services               Xiaomi

R1-2109520         Remaining issues on evaluation methodology for XR Samsung

R1-2109552         Remaining issues on evaluation methodology for XR and CG  MediaTek Inc.

R1-2109639         On remaining issues of evaluation methodology for XR            Intel Corporation

R1-2109706         Discussion on evaluation methodology for XR            NTT DOCOMO, INC.

R1-2109738         Development of the Evaluation Methodology for XR Study     Nokia, Nokia Shanghai Bell

R1-2109925         Remaining Issues on XR Evaluation Methodology     InterDigital, Inc.

R1-2109989         Remaining issues on evaluation methodology for XR LG Electronics

R1-2110061         Remaining issues on XR evaluation methodology       Apple

R1-2110217         Remaining Issues on Evaluation Methodology for XR              Qualcomm Incorporated

[106bis-e-NR-XR-02] – Xiaohang (vivo)

Email discussion/approval on remaining issues on XR evaluation methodology

-        1^st check point: October 14

-        Final check point: October 19

R1-2110485        Summary #1 on remaining issues of traffic model and evaluation methodology for XR   Moderator (vivo)

R1-2110553        Summary #2 on remaining issues of traffic model and evaluation methodology for XR   Moderator (vivo)

From Oct 18^th GTW session

Working Assumption

XR mobility performance is evaluated analytically taking into account mobility procedures, agreed traffic models, and user satisfaction criteria. Following methodology is adopted

·        Alternative 1 (Modified Option 3):

o     For XR/Cloud Gaming mobility evaluation, the metric is defined to be where N is the number of consecutive XR packets lost due to a HO event and T is the minimum target time interval between HO events, which are obtained by the following steps

§  Step 1. HO interruption time is calculated for existing HO techniques by directly following the requirements given in 3GPP TS 38.133.

§  Step 2. For a HO interruption time Y (calculated in Step 1) and the XR traffic pattern characterized by the inter-arrival time in average R and the packet delay budget PDB:

·        Number of consecutive XR packets lost due to a HO event, N is estimated as: N = (Y – PDB) / R

·        Minimum target time interval between HO events, T is estimated as:

o       where  is packet error rate during time outside of handover procedure. Companies can report the value of   used in the evaluation and assumptions.

o   X is the UE satisfactory requirement (baseline: X = 99%, other X value(s) can be also evaluated).

·        Note 1: how to draw the observations/conclusion based on the simplified assumption will be discussed in RAN1 #107e.

·        Note 2: mobility evaluation is performed in dense Urban and UMA

·        Note 3: T maybe affected by system load, interference, etc.

Decision: As per email decision posted on Oct 22^nd,

Agreement 

XR mobility performance is evaluated analytically taking into account mobility procedures, agreed traffic models, and user satisfaction criteria. Following methodology is adopted

·        Alternative 1 (Modified Option 3):

o   For XR/Cloud Gaming mobility evaluation, the metric is defined to be where N is the number of consecutive XR packets lost due to a HO event and T is the minimum target time interval between HO events, which are obtained by the following steps

§  Step 1. HO interruption time is calculated for existing HO techniques by directly following the requirements given in 3GPP TS 38.133.

§  Step 2. For a HO interruption time Y (calculated in Step 1) and the XR traffic pattern characterized by the inter-arrival time packet arrival rate in average R and the packet delay budget PDB:

·        Number of consecutive XR packets lost due to a HO event, N is estimated as: N = (Y – PDB) * R, Y >= PDB

·        Minimum target time interval between HO events, T is estimated as:

o   where  is packet error rate during time outside of handover procedure. Companies can report the value of  used in the evaluation and assumptions.

o   X is the UE satisfactory requirement (baseline: X = 99%, other X value(s) can be also evaluated).

·        Company can optionally evaluate the case of Y < PDB. E.g. N = max {(Y – PDB) * R, 0}, and , when Y < PDB; Or N = Y * R, and , when Y < PDB.

·        Note 1: how to draw the observations/conclusion based on the simplified assumption will be discussed in RAN1 #107e.

·        Note 2: mobility evaluation is performed in dense Urban and UMA

·        Note 3: T maybe affected by system load, interference, etc.

Note: above Working Assumption made in this meeting (from Oct 18^th) does not need confirm anymore.

Final summary in:

R1-2110675        Summary on [106bis-e-NR-XR-02] Email discussion/approval on remaining issues on XR evaluation methodology        Moderator (vivo)

8.14.33     Other

R1-2108891         On Capacity and Power Working Areas for XR           ZTE, Sanechips

R1-2109010         Potential enhancement areas for XR              vivo

R1-2109102         Discussion on support of XR/CG service      OPPO

R1-2109112         Discussion on enhancements for XR              Ericsson

R1-2109137         Potential enhancements of NR to support XR services              NEC

R1-2109199         Potential area of NR enhancement for the support of XR services           CATT

R1-2109395         Discussion on potential enhancement for supporting XR           Xiaomi

R1-2109521         Potential enhancements for XR       Samsung

R1-2109556         Further Potential XR Enhancements              MediaTek Inc.

R1-2109739         Enhancements for better XR support over NR             Nokia, Nokia Shanghai Bell

R1-2109745         Challenges and potential enhancements for XR and Cloud Gaming        Huawei, HiSilicon

R1-2109803         Discussion on potential enhancements for XR             Sony

R1-2109926         Discussion on potential enhancements for XR             InterDigital, Inc.

R1-2110519         Potential enhancements for XR in Rel-18     Apple    (rev of R1-2110062)

R1-2110218         Potential Enhancements for XR       Qualcomm Incorporated

 RAN1#107-e

8.14   Study on XR Evaluations for NR (RAN1)

Please refer to RP-210460 for detailed scope of the WI.

R1-2112799        Session notes for 8.14 (Study on XR Evaluations for NR)    Ad-Hoc Chair (CMCC)

8.14.1     Performance Evaluation Results

R1-2110811         Performance evaluation results for XR and Cloud Gaming       Huawei, HiSilicon

R1-2110885         XR evaluation results        FUTUREWEI

R1-2111046         Performance evaluation results for XR          vivo

R1-2111234         Evaluation results of XR performance           CATT

R1-2111349         Evaluation results for XR evaluation             OPPO

R1-2111351         Performance Evaluation Results for XR        ZTE, Sanechips

R1-2111360         Evaluation Results for XR CEWiT

R1-2111521         Performance evaluation results for XR and CG           Intel Corporation

R1-2112573         Performance evaluation result for XR            Xiaomi  (rev of R1-2111556)

R1-2111632         Performance evaluation results for XR          CMCC

R1-2111806         XR Performance Evaluation Results              AT&T

R1-2112572         Performance results in indoor hotspot and dense urban deployments of CG and VR/AR applications          Nokia, Nokia Shanghai Bell            (rev of R1-2111828)

R1-2111830         Performance Evaluation Results for XR        InterDigital, Inc.

R1-2111902         Performance evaluation on XR        Apple

R1-2112069         Performance evaluation results for XR          LG Electronics    (Late submission)

R1-2112079         Performance Evaluation Results for XR        China Unicom

R1-2112551         XR performance evaluation results Ericsson (rev of R1-2112160)

R1-2112175         Performance evaluation results of XR            ITRI

R1-2112720         Evaluation Results for XR Qualcomm Incorporated   (rev of R1-2112648, rev of R1-2112530, rev of R1-2112244)

R1-2112296         Performance Evaluation Results for XR        MediaTek Inc.

From Nov 11^th GTW session

Conclusion

Source XX, [reference number] (Note: without attaching company name) will be used in observations

Conclusion

For the results from companies to be captured in the TR, use only “observations” which is captured in the current version that has been uploaded to ftp server.

·        How to capture enhancement related results?

·        No further discussion on the traffic model

From Nov 15^th GTW session

Conclusion

Remove the statements of benefit of enhancement from the “description” part for all the enhancements.

Conclusion

Split Delay Aware/Frame Level Integrated Transmission Scheduler of section 8.3.3.2 of TR Section – TR Capacity into two sections.

[107-e-NR-XR-01] – Xiaohang (vivo)

Email discussion/approval on XR performance evaluation results for capacity

-        1^st check point: November 15

-        Final check point: November 19

R1-2112695        Summary #1 of [107-e-NR-XR-01] discussion on XR performance evaluation results for capacity          Moderator (vivo)

R1-2112668        [DRAFT] TR section - Capacity evaluation             Moderator (vivo)

R1-2112722        [DRAFT#2] TR section - Capacity evaluation         Moderator (vivo)

Decision: From Nov 19^th GTW session, R1-2112722 is endorsed in principle for inclusion into TR

[107-e-NR-XR-02] – Eddy (Qualcomm)

Email discussion/approval on XR performance evaluation results for power

-        1^st check point: November 15

-        Final check point: November 19

R1-2112705         XR power evaluation Section in TR 38.838  Moderator (Qualcomm)

From Nov 17^th GTW session

DRAFT TR section for Power Evaluation-v021_FL_clean is endorsed in principle, with following modifications:

·        Remove “[“ in beginning of  section 9.3.3.7 UL active time

·        Romove “to make sure that UE will end skipping before the lower jitter boundary of the next packet” , and “a conservative ” of section 9.3.1 Baseline Power Evaluation Results

·        Add (s) after “skipping duration” in section 9.3.1 Baseline Power Evaluation Results

R1-2112809        TR section for Power Evaluation Moderator (Qualcomm)

Decision: From Nov 19^th GTW session, R1-2112809 is endorsed in principle for inclusion into TR

[107-e-NR-XR-03] – Eddy (Qualcomm)

Email discussion/approval on XR performance evaluation results for coverage

-        1^st check point: November 15

-        Final check point: November 19

R1-2112704        XR coverage evaluation Section in TR 38.838         Moderator (Qualcomm)

R1-2112810        TR section for Coverage Evaluation           Moderator (Qualcomm)

Decision: From Nov 19^th GTW session, R1-2112810 is endorsed in principle for inclusion into TR

[107-e-NR-XR-04] – Xiaohang (vivo)

Email discussion/approval on XR performance evaluation results for mobility

-        1^st check point: November 15

-        Final check point: November 19

R1-2112696        Summary #1 of [107-e-NR-XR-04] discussion on XR performance evaluation results for mobility          Moderator (vivo)

R1-2112669        [DRAFT] TR section - Mobility evaluation              Moderator (vivo)

R1-2112723        [DRAFT#2] TR section - Mobility evaluation          Moderator (vivo)

Decision: From Nov 19^th GTW session, R1-2112723 is endorsed in principle for inclusion into TR

[107-e-NR-XR-05] – Eddy (Qualcomm)

Email discussion/approval on conclusion section of TR 38.838

-        1^st check point: November 15

-        Final check point: November 19

R1-2112821        Conclusion section in TR 38.838  Moderator (Qualcomm)

Decision: From Nov 19^th GTW session, R1-2112821 is endorsed in principle with corresponding modifications based on below agreements, to be further incorporated into TR

Agreement

The capacity for AR, VR, and CG applications was evaluated and the results are summarized as follows:

·        The baseline capacity for AR, VR, and CG in FR1 DL/UL and FR2 DL/UL were evaluated based on the agreed traffic models, evaluation methodology, and KPIs, with the results collected in Clause 8.3.1. The evaluation results show that

o   Option 2: 5G NR can support AR, VR, and CG for the evaluated cases and scenarios, where the capacity in urban macro scenario is generally lower than that in dense urban and indoor hotspot scenarios, in particular for AR applications with uplink video.

Agreement

Based on the study, it is recommended to further study and enhance at least the capacity and UE power consumption performance of 5G NR for XR and CG applications.

Note: No intention to decide the priority of capacity and UE power consumption in future normative work or study.

R1-2112812        TR 38.838 v0.2.0               Moderator (Qualcomm)

Decision: From Nov 19^th GTW session, R1-2112812 is endorsed as v0.2.0 as baseline for v1.0.0 for one step approval by RAN plenary.

MCC post-meeting: the endorsed v0.2.0 is not using proper template and not based on the latest v0.1.0 uploaded to 3GU.

8.14.22     Other

Including any remaining issues on evaluation methodology.

R1-2111047         Potential enhancement areas for XR              vivo

R1-2111235         Potential area of NR enhancement for the support of XR services           CATT

R1-2111350         Discussion on support of XR/CG service      OPPO

R1-2111352         Considerations on Capacity and Power Working Areas for XR ZTE, Sanechips

R1-2111409         Potential enhancements for XR       Sony

R1-2111522         Potential enhancement areas for XR              Intel Corporation

R1-2111690         Potential enhancements of NR to support XR services              NEC

R1-2111766         Potential enhancements for XR       Samsung

R1-2111787         Discussion on enhancements for XR              Ericsson

R1-2111829         Enhancements for better XR support over NR             Nokia, Nokia Shanghai Bell

R1-2111831         Discussion on remaining issues and enhancements for XR evaluations   InterDigital, Inc.

R1-2111903         Views on enhancements for XR in Rel-18    Apple

R1-2112245         Potential Enhancements for XR       Qualcomm Incorporated

R1-2112299         Further Potential XR Enhancements              MediaTek Inc.

R1-2112405         TP for Conclusions of TR 38.838    Huawei, HiSilicon